Spatio-Temporal Video Object Segmentation via Scale-Adaptive 3D Structure Tensor

To address multiple motions and deformable objects’ motions encountered in existing region-based approaches, an automatic video object (VO) segmentation methodology is proposed in this paper by exploiting the duality of image segmentation and motion estimation such that spatial and temporal information could assist each other to jointly yieldmuch improved segmentation results. The key novelties of ourmethod are (1) scale-adaptive tensor computation, (2) spatial-constrained motionmask generation without invoking densemotion-field computation, (3) rigidity analysis, (4) motionmask generation and selection, and (5) motionconstrained spatial region merging. Experimental results demonstrate that these novelties jointly contribute much more accurate VO segmentation both in spatial and temporal domains.